Method and apparatus for content association and history tracking in virtual and augmented reality

ABSTRACT

A machine-implemented method includes establishing a virtual or augmented reality entity, and establishing a state for the entity having a state time and state properties including a state spatial arrangement. The data entity and state are stored, and are subsequently received and outputted at a time other than the state time so as to exhibit a “virtual time machine” functionality. An apparatus includes a processor, a data store, and an output. A data entity establisher, a state establisher, a storer, a data entity receiver, a state receiver, and an outputter are instantiated on the processor.

CLAIM OF PRIORITY

This application is a Continuation Application of U.S. patentapplication Ser. No. 14/014,296, entitled “METHOD AND APPARATUS FORCONTENT ASSOCIATION AND HISTORY TRACKING IN VIRTUAL AND AUGMENTEDREALITY,” filed Aug. 29, 2013, which claims the benefit of U.S.Provisional Application Ser. No. 61/695,268 filed on Aug. 30, 2012, thecontents of which are incorporated by reference for all intents andpurposes.

FIELD OF THE INVENTION

This invention relates to association of content in virtual and/oraugmented reality environments. More particularly, the invention relatesto establishing historical states for virtual and/or augmented realityentities, so as to enable at least partial reconstruction of entitiesand/or environments as at earlier times.

DESCRIPTION OF RELATED ART

In some virtual and/or augmented reality environments, entities may bedisposed therein that a user of such environments may view, manipulate,modify, etc. Depending on the particulars of a given environment, usersmay be able to move or rotate virtual/augmented objects, cause suchobjects to appear and disappear, change color, etc.

However, facilitating such malleability in a virtual or augmentedreality has drawbacks. Namely, in altering a virtual or augmentedreality entity, a user is in some sense destroying or overwriting theoriginal entity. The original state of the entity, and/or intermediatestates of the entity, may be lost. While this can pose difficulties foreven a simple environment supporting a single user, it may becomeincreasingly problematic as the size, complexity, and number of entitiesincreases, and as the number of potential users increases. Inparticular, for a large shared environment with many users, it ispossible for even a single such user (whether by accident or malice) toirreparably alter or destroy entities important to the environmentand/or the experience that environment provides.

There is a need for a method and apparatus to support association ofcontent for virtual and/or augmented reality environments in such a wayas to oppose undesirable loss of such content, without unduly limitingthe ability of users to manipulate content.

BRIEF SUMMARY OF THE INVENTION

The present invention contemplates a variety of systems, apparatus,methods, and paradigms for associating virtual and/or augmented realitycontent.

In one embodiment of the present invention, a machine-implemented methodis provided. The method includes establishing a data entity, the dataentity being an augmented reality entity and/or a virtual realityentity. The method also includes establishing a state for the dataentity, said state including a state time and state properties that atleast substantially correspond to properties of the data entitysubstantially at the time of establishment of the state, at least one ofthe plurality of state properties being a state spatial arrangement ofsaid the entity. The method further includes storing the data entity andthe state so as to enable output of the data entity to an augmentedreality environment and/or a virtual reality environment, with the dataentity exhibiting at least one state property as at the state time, theoutput of the data entity being at an output time substantiallydifferent from the state time.

Establishing data entity may include selecting the data entity, the dataentity being in existence prior to the selection thereof. Establishingthe data entity may include creating the data entity. Establishing thedata entity may include generating the data entity from a parent entity.The parent entity may be a physical object, a physical background, aphysical environment, a physical creature, and/or a physical phenomenon.The parent entity may be an augmented reality object and/or a virtualreality object.

At least a portion of the state may be incorporated within the dataentity. At least a portion of the state may be distinct from the dataentity.

The state properties may include the identity of the data entity.

The state spatial arrangement may include an absolute position of thedata entity and/or an absolute orientation of the data entity. The statespatial arrangement may include a relative position of the data entityand/or a relative orientation of the data entity.

The state properties may include a still image, a video, audio,olfactory data, a 2D model, a 3D model, text, numerical data, anenvironmental condition, animation, resolution, frame rate, bit depth,sampling rate, color, color distribution, spectral signature,brightness, brightness distribution, reflectivity, transmissivity,absorptivity, surface texture, geometry, mobility, motion, speed,direction, acceleration, temperature, temperature distribution,composition, chemical concentration, electrical potential, electricalcurrent, mass, mass distribution, density, density distribution, price,quantity, nutritional information, user review, presence, visibility,RFID data, barcode data, a file, executable instructions, a hyperlink, adata connection, a communication link, contents, an association, acreator, and/or a system ID.

Establishing the data entity may include comprises distinguishing thedata entity from its surroundings.

The method may include retrieving the data entity, retrieving the state,and at an output time substantially different from the state timeoutputting the data entity to an augmented reality environment and/orvirtual reality environment, with the data entity exhibiting at leastone of the state properties as at the state time.

The method may include establishing multiple data entities andestablishing a state corresponding with each data entity, the statecomprising a state time and state properties at least substantiallycorresponding to properties of the data entity substantially at the timeof establishment of the state, the state properties including a statespatial arrangement of the data entity. The method may further includestoring each data entity and its corresponding state so as to enableoutput of the entities to the augmented reality environment and/orvirtual reality environment with each data entity exhibiting at leastone of the corresponding state properties as at the corresponding statetime, the output of the data entities being at an output timesubstantially different from the state times.

The method may include establishing multiple states for the data entity,each state including a state time and state properties at leastsubstantially corresponding to properties of the data entitysubstantially at the time of establishment of the state, the stateproperties including a state spatial arrangement of the data entity. Themethod may further include storing the data entity and the states so asto enable output of the data entity to the augmented reality environmentand/or virtual reality environment with the data entity exhibiting atleast one state property for a selected state as at the state time forthe selected state, the output of the data entity being at an outputtime substantially different from the state time for the selected state.

In another embodiment of the present invention, an apparatus is providedthat includes means for establishing a data entity, the data entitycomprising at least one of a group consisting of an augmented realityentity and a virtual reality entity, means for establishing a state forthe data entity, the state including a state time and state propertiesat least substantially corresponding to properties of the data entitysubstantially at the time of establishment of the state, the pluralityof state properties comprising a state spatial arrangement of the dataentity. The apparatus also includes means for storing the data entityand the state so as to enable output of the data entity to an augmentedreality environment and/or virtual reality environment with the dataentity exhibiting at least one of the state properties as at the statetime, the output of the data entity being at an output timesubstantially different from the state time.

In another embodiment of the present invention, a machine-implementedmethod is provided, the method including receiving a data entity, thedata entity being an augmented reality entity and/or a virtual realityentity. The method includes receiving a state for the data entity, thestate including a state time and state properties at least substantiallycorresponding to properties of the data entity substantially at thestate time, the state properties including a state spatial arrangementof the data entity. The method further includes, at an output timesubstantially different from the state time, outputting the data entityto an augmented reality environment and/or a virtual realityenvironment, the data entity exhibiting at least one of the stateproperties as at the state time.

At least a portion of the state may be incorporated within the dataentity. At least a portion of the state may be distinct from the dataentity.

The state properties may include an identity of the data entity.

The state spatial arrangement may include an absolute position of thedata entity and/or an absolute orientation of the data entity. The statespatial arrangement may include a relative position of the data entityand/or a relative orientation of the data entity.

The state properties may include a still image, a video, audio,olfactory data, a 2D model, a 3D model, text, numerical data, anenvironmental condition, animation, resolution, frame rate, bit depth,sampling rate, color, color distribution, spectral signature,brightness, brightness distribution, reflectivity, transmissivity,absorptivity, surface texture, geometry, mobility, motion, speed,direction, acceleration, temperature, temperature distribution,composition, chemical concentration, electrical potential, electricalcurrent, mass, mass distribution, density, density distribution, price,quantity, nutritional information, user review, presence, visibility,RFID data, barcode data, a file, executable instructions, a hyperlink, adata connection, a communication link, contents, an association, acreator, and/or a system ID.

The method may include receiving a first state for the data entity, thefirst state including a first state time and first state properties atleast substantially corresponding to properties of the data entitysubstantially at the first state time, the first state propertiesincluding a first state spatial arrangement of the data entity, andreceiving a second state for the data entity, the second state includinga second state time and second state properties at least substantiallycorresponding to properties of the data entity substantially at thesecond state time, the second state properties including a second statespatial arrangement of the data entity. The method may further include,at the output time, outputting a first iteration of the data entityexhibiting at least one of the first state properties as at the firststate time, at the output time outputting a second iteration of the dataentity exhibiting at least one of the second state properties as at thesecond state time.

The method may include receiving a first state for the data entity, thefirst state including a first state time and first state properties atleast substantially corresponding to properties of the data entitysubstantially at the first state time, the first state propertiesincluding a first state spatial arrangement of the data entity, andreceiving a second state for the data entity, the second state includinga second state time and second state properties at least substantiallycorresponding to properties of the data entity substantially at thesecond state time, the second state properties including a second statespatial arrangement of the data entity. The method may furtherincluding, at a first output time substantially different from the firststate time, outputting a first iteration of the data entity exhibitingat least one first state property as at the first state time, and at asecond output time subsequent to the first output time, outputting asecond iteration of the data entity exhibiting at least one second stateproperty as at the second state time.

The data entity may exhibits at least the state spatial arrangement asat the state time. The data entity may not exhibit the state spatialarrangement as at the state time. The data entity may exhibit all of thestate properties as at the state time.

In another embodiment of the present invention, an apparatus is providedincluding means for receiving a data entity, the data entity being anaugmented reality entity and/or a virtual reality entity, and means forreceiving a state for the data entity, the state including a state timeand state properties at least substantially corresponding to propertiesof the data entity substantially at the state time, the state propertiesincluding a state spatial arrangement of the data entity. The apparatusfurther includes means for, at an output time substantially differentfrom the state time, outputting the data entity to an augmented realityenvironment and/or a virtual reality environment, the data entityexhibiting at least one of the state properties as at the state time.

In another embodiment of the present invention, an apparatus is providedthat includes a processor adapted to execute executable instructions. Adata entity establisher is instantiated on the processor, the dataentity establisher including executable instructions adapted forestablishing a data entity that is an augmented reality entity and/or avirtual reality entity. A state establisher is instantiated on theprocessor, the state establisher including executable instructionsadapted for establishing a state for the data entity, the stateincluding a state time and state properties at least substantiallycorresponding to properties of the data entity substantially at thestate time, the state properties including a state spatial arrangementof the data entity. The apparatus includes a data store in communicationwith the processor. A storer also is instantiated on the processor, thestorer including executable instructions adapted for storing in the datastore the data entity and the state so as to enable output of the dataentity to an augmented reality environment and/or a virtual realityenvironment with the data entity exhibiting at least one state propertyas at the state time, the output of the data entity being at an outputtime substantially different from the state time.

The apparatus may include a chronometer adapted to determine the outputtime. The apparatus may include at least one sensor in communicationwith the processor. The at least one sensor may be adapted to determinethe spatial arrangement of the apparatus at the state time, the spatialarrangement of the apparatus at the output time, and/or the statespatial arrangement of the data entity.

The sensor may be an accelerometer, a gyroscope, an imager, a stereopair of imagers, a GPS sensor, a magnetometer, a structured lightsensor, a time-of-flight sensor, an ultrasonic sensor, and/or a wirelesssignal triangulation sensor.

The sensor be a bar code reader, a chemical sensor, an electricalsensor, an electrical field sensor, a gas detector, a humidity sensor,an imager, a stereo pair of imagers, a light sensor, a magnetic fieldsensor, a microphone, a motion sensor, a pressure sensor, a radarsensor, a radiation sensor, an RFID sensor, a smoke sensor, aspectrometer, a thermal sensor, an ultrasonic sensor, and/or a vibrationsensor.

The apparatus may include a data entity distinguisher instantiated onthe processor, the data entity distinguisher including executableinstructions adapted for distinguishing the data entity from asurrounding thereof.

The apparatus may include an identifier instantiated on the processor,the data entity identifier including executable instructions adapted foridentifying the data entity, a parent entity for the data entity, and/orat least one of the state properties.

Some or all of the apparatus may be disposed on a head mounted display.

The apparatus may include a data entity receiver instantiated on theprocessor, the data entity receiver including executable instructionsadapted for receiving the data entity from the data store. The apparatusalso may include a state receiver instantiated on the processor, thestate receiver including executable instructions adapted for receivingfrom the data store the state for the data entity. The apparatus mayfurther include an output in communication with the processor, theoutput being adapted to output the data entity. The output also mayinclude an outputter instantiated on the processor, the outputterincluding executable instructions adapted for outputting the data entityan augmented reality environment and/or a virtual reality environment atan output time substantially different from the state time, with thedata entity exhibiting at least one of the state properties as at thestate time.

In another embodiment of the present invention, an apparatus is providedthat includes a processor adapted to execute executable instructions anda data store in communication with the processor. A data entity receiveris instantiated on the processor, the data entity receiver includingexecutable instructions adapted for receiving from the data store a dataentity, the data entity including an augmented reality entity and/or avirtual reality entity. A state receiver is instantiated on theprocessor, the state receiver including executable instructions adaptedfor receiving from the data store a state for the data entity, the stateincluding a state time and state properties at least substantiallycorresponding to properties of the data entity substantially at thestate time, the state properties including a state spatial arrangementof the data entity. The apparatus includes an output in communicationwith the processor, the output being adapted to output the data entity.An outputter is instantiated on the processor, the outputter includingexecutable instructions adapted for outputting the data entity to anaugmented reality environment and/or a virtual reality environment at anoutput time substantially different from the state time, with the dataentity exhibiting at least one of the state properties as at the statetime.

The apparatus may include a chronometer adapted to determine the outputtime.

The apparatus may include at least one sensor in communication with theprocessor. The sensor may be adapted to determine the spatialarrangement of the apparatus at the state time, the spatial arrangementof the apparatus at the output time, and/or the state spatialarrangement of the data entity. The sensor may be an accelerometer, agyroscope, an imager, a stereo pair of imagers, a GPS sensor, amagnetometer, a structured light sensor, a time-of-flight sensor, anultrasonic sensor, and/or a wireless signal triangulation sensor.

16. The output may be a visual display. The output may be a stereo pairof visual displays.

Some or all of the apparatus may be disposed on a head mounted display.

In another embodiment of the present invention, a method is providedthat includes instantiating on a processor a data entity establisher,the data entity establisher including executable instructions adaptedfor establishing a data entity that is an augmented reality entityand/or a virtual reality entity. The method includes instantiating onthe processor a state establisher, the state establisher includingexecutable instructions adapted for establishing a state for the dataentity, the state including a state time and a state properties at leastsubstantially corresponding to properties of the data entitysubstantially at the state time, the state properties including a statespatial arrangement of the data entity. The method also includesinstantiating on the processor a storer, the storer including executableinstructions adapted for storing the data entity and the state so as toenable output of the data entity to an augmented reality environmentand/or a virtual reality environment with the data entity exhibiting atleast one of the state properties as at the state time, the output ofthe data entity being at an output time substantially different from thestate time.

The data entity establisher may be adapted to select the data entity,the data entity being in existence prior to a selection thereof. Thedata entity establisher may be adapted to create the data entity. Thedata entity establisher may be adapted to generate the data entity froma parent entity. The parent entity may be a physical object, a physicalbackground, a physical environment, a physical creature, and/or aphysical phenomenon. The parent entity may be an augmented realityobject and/or a virtual reality object.

The method may include instantiating on the processor a data entitydistinguisher, the data entity distinguisher including executableinstructions adapted for distinguishing the data entity from asurrounding thereof.

The method may include instantiating on the processor an identifier, theidentifier including executable instructions adapted for identifying thedata entity, a parent entity for the data entity, and/or at least one ofthe state properties.

The method may include instantiating on the processor a data entityreceiver, the data entity receiver including executable instructionsadapted for receiving the data entity from the data store. The methodmay include instantiating on the processor a state receiver, the statereceiver including executable instructions adapted for receiving thestate from the data store. The method may also include instantiating onthe processor an outputter, the outputter including executableinstructions adapted for outputting the data entity to an augmentedreality environment and/or a virtual reality environment at the outputtime, with the data entity exhibiting at least one of the plurality ofstate properties as at the state time.

In another embodiment of the present invention a method is provided thatincludes instantiating on a processor a data entity receiver, the dataentity receiver including executable instructions adapted for receivinga data entity from a data store, the data entity including an augmentedreality entity and/or a virtual reality entity. The method includesinstantiating on the processor a state receiver, the state receiverincluding executable instructions adapted for receiving from the datastore a state for the data entity, the state including a state time andstate properties at least substantially corresponding to properties ofthe data entity substantially at the state time, the state propertiesincluding a state spatial arrangement of the data entity. The methodalso includes instantiating on the processor an outputter, the outputterincluding executable instructions adapted for outputting the data entityto an augmented reality environment and/or a virtual reality environmentat an output time substantially different from the state time, with thedata entity exhibiting at least one of the state properties as at thestate time.

The state receiver may be adapted to receive from the data store a firststate for the data entity, the first state including a first state timeand first state properties at least substantially corresponding toproperties of the data entity substantially at the first state time, thefirst state properties including a first state spatial arrangement ofthe data entity. The state receiver may also be adapted to receive fromthe data store a second state for the data entity, the second stateincluding a second state time and second state properties at leastsubstantially corresponding to properties of the data entitysubstantially at the second state time, the second state propertiesincluding a second state spatial arrangement of the data entity. Theoutputter may be adapted to output at the output time a first iterationof the data entity exhibiting at least one of the first state propertiesas at the first state time, and the outputter may be adapted to outputat the output time a second iteration of the data entity exhibiting atleast one of the second state properties as at the second state time.

The state receiver may be adapted to receive from the data store a firststate for the data entity, the first state including a first state timeand a first state properties at least substantially corresponding toproperties of the data entity substantially at the first state time, thefirst state properties including a first state spatial arrangement ofthe data entity. The state receiver also may be adapted to receive fromthe data store a second state for the data entity, the second stateincluding a second state time and second state properties at leastsubstantially corresponding to properties of the data entitysubstantially at the second state time, the second state propertiesincluding a second state spatial arrangement of the data entity. Theoutputter may be adapted to output at a first output time substantiallydifferent from the first state time a first iteration of the data entityexhibiting at least one of the first state properties as at the firststate time. The outputter also may be adapted to output at a secondoutput time subsequent to the first output time a second iteration ofthe data entity exhibiting at least one of second state properties as atthe second state time.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Like reference numbers generally indicate corresponding elements in thefigures.

FIG. 1 shows an example method for associating content for an entityaccording to the present invention.

FIG. 2 shows an example method for outputting associated content for anentity according to the present invention.

FIG. 3 shows an example method for associating content for an entity andoutputting the associated content for the entity according to thepresent invention.

FIG. 4 shows an example method for associating content for multipleentities according to the present invention.

FIG. 5 shows an example method for associating content for multiplestates for an entity according to the present invention.

FIG. 6 shows an example method for outputting associated content formultiple states of an entity in multiple iterations thereof according tothe present invention.

FIG. 7 shows an example method for outputting associated content formultiple states of an entity in a multiple iterations at different timesaccording to the present invention.

FIG. 8 shows a functional diagram of an example embodiment of anapparatus for associating content for an entity according to the presentinvention.

FIG. 9 shows a functional diagram of an example embodiment of anapparatus for outputting associated content for an entity according tothe present invention.

FIG. 10 shows a functional diagram of an example embodiment of anapparatus for associating content for an entity and outputtingassociated content for the entity according to the present invention.

FIG. 11 shows a functional diagram of an example embodiment of anapparatus for associating content for an entity according to the presentinvention, with capabilities for distinguishing and identifying content.

FIG. 12 shows a functional diagram of an example embodiment of anapparatus for associating content for an entity according to the presentinvention, with a chronometer and sensor.

FIG. 13 shows a functional diagram of an example embodiment of anapparatus for outputting associated content for an entity according tothe present invention, with capabilities for distinguishing andidentifying content.

FIG. 14. shows a functional diagram of an example embodiment of anapparatus for outputting associated content for an entity according tothe present invention, with a chronometer and sensor.

FIG. 15 shows a functional diagram of an example embodiment of anapparatus for associating content for an entity and outputtingassociated content for the entity according to the present invention,with capabilities for distinguishing and identifying content, and with achronometer and sensor.

FIG. 16 shows an example method for establishing capabilities forassociating content for an entity according to the present inventiononto a processor.

FIG. 17 shows an example method for establishing capabilities foroutputting associated content for an entity according to the presentinvention onto a processor.

FIG. 18 shows an example method for establishing capabilities forassociating content for an entity and outputting the associated contentfor the entity according to the present invention onto a processor.

FIG. 19 shows an example method for establishing capabilities forassociating content for an entity according to the present inventiononto a processor, along with capabilities for distinguishing andidentifying content.

FIG. 20 shows an example method for establishing capabilities foroutputting associated content for an entity according to the presentinvention onto a processor, along with capabilities for distinguishingand identifying content.

FIG. 21 shows a perspective view of an example embodiment of anapparatus for associating content for an entity and/or outputtingassociated content for the entity according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, an example machine-controlled method forassociating content for an entity according to the present invention isshown therein.

In the method shown in FIG. 1, a data entity is established 102 in aprocessor. Some explanation of “data entity” and “established” withregard to the present invention may illuminating prior to furtherdescription of the example method.

With regard to the present invention, the term “data entity” encompassesinformation constructs that represent one or more objects, elements,phenomena, locales, environments, etc. Alternately, wherein an object,phenomenon, etc. that is to be represented is itself an informationconstruct, e.g. a digital image or 3D model, the data entity may itselfbe either the original information construct or a copy thereof. However,it is emphasized that even in such cases, it is not required that a dataentity be an exact or even an approximate copy of an originalinformation construct.

A data entity may be a virtual reality entity and/or may be an augmentedreality entity. For example, a virtual avatar, an augmented markingidentifying a real-world bus stop, a review of a restaurant attached tothe sidewalk in front of the restaurant as an augmentation, etc. may allbe treated as data entities for purposes of the present invention.

However, the concept of a data entity for purposes of the presentinvention is not limited only to representing well-defined or individualobjects. For example, a data construct representing the collectiveimage, appearance, 3D model, etc. of a background at some location maybe considered a data entity for at least certain embodiments of thepresent invention; although such a background may not be, in a strictsense, an “object”, nevertheless for at least certain embodiments abackground can be reasonably represented as a single data entity.Likewise, other groupings and even abstractions may be considered and/orrepresented as data entities, e.g. “foot traffic” might be treatedcollectively as a data entity, for example being record of the number ofpersons walking through an area, the average speed of the persons, theproportion of persons walking one direction as opposed to anotherdirection, etc. Given such an arrangement, the individual persons wouldnot necessarily be represented with individual entities therefor (thoughrepresenting individuals with individual entities is also not excluded);nevertheless an entity may be established to represent the collectivephenomenon of foot traffic.

It is emphasized that although a data entity for purposes of the presentinvention is, as noted, an information construct, data entities mayderive from parent entities that are real-world objects, elements,phenomena, locales, environments, etc. Thus, although a data entity maybe data, such a data entity may represent, and may even be visuallyindistinguishable from, one or more actual physical features. Forexample, a data entity might represent a physical automobile, a portionof a physical automobile, a group of physical automobiles, etc.

The concept of establishing a data entity also is to be consideredbroadly with regard to the present invention. It is noted that to“establish” something may, depending on particulars, refer to either orboth the creation of something new (e.g. establishing a business,wherein a new business is created) and the determination of a conditionthat already exists (e.g. establishing the whereabouts of a person,wherein the location of a person who is already present at that locationis discovered, received from another source, etc.). Similarly,establishing a data entity may encompass several potential approaches,including but not limited to the following.

Establishing a data entity may include generating the data entity fromsome parent entity, including but not limited to a physical object, avirtual object, an augmented object, or some other data object.

Establishing a data entity also may include creating the data entitywithout regard to a parent entity, e.g. a processor may executeinstructions so as to create a data entity in some fashion, whether fromexisting data, user inputs, internal algorithms, etc.

Establishing a data entity additionally may include selecting apreviously-existing data entity, for example by reading a data entityfrom a data store, downloading a data entity from a communication link,or otherwise obtaining a data entity that already exists substantiallyin a form as to be used by some embodiment of the present invention.

The present invention is not particularly limited insofar as how a dataentity may be established. It is required only that a data entity thatis functional in terms of the present invention is in some fashion mademanifest. Other arrangements than those described may be equallysuitable. Also, where used with regard to other steps such asestablishing a state, establishing a state time, establishing a stateproperty, etc., establishing should be similarly be interpreted in abroad fashion.

Returning to the method of FIG. 1, a state is established 104 for thedata entity. For illustrative purposes, establishing a state 104 for thedata entity may be considered as two sub-steps, as described below.However, it is emphasized with regard to establishing a state 104 thatin at least some embodiments the sub-steps may be executed together,simultaneously, in parallel, etc., and are not necessarily required tobe executed as distinct steps.

In addition, it is noted that although for certain embodiments it may beuseful to establish a state for a data entity immediately uponestablishing the data entity itself, e.g. as a “baseline” state for thatdata entity, this is not required. Establishment of a data entity andestablishment of a state therefor are distinct events, and may beexecuted at different times, in different places, using differentprocessors, by different users, etc. Indeed, in general individual stepsof the example methods as shown and described should not be consideredto necessarily be limited to sequential execution, simultaneousexecution, etc. unless so specified herein or so mandated by logic.

Returning to FIG. 1, in establishing a state 104, a state time isestablished 104A, and state properties also are established 104B for thedata entity, the state properties at least substantially correspondingto properties of the data entity as of the state time.

A state time is a time reference for when the state is established. Thatis, a state that is established at 3:02 PM on Jul. 19 of 2014 would havethat time as a state time. It is preferable that a state time besufficiently precise and/or accurate as to uniquely identify aparticular state for a particular data entity. However, no specificlimits regarding precision, accuracy, format, etc. are imposed, and suchfactors may vary from one embodiment to another.

A state property is some feature of the data entity itself or that is ormay be associated with the data entity, and/or that provides some degreeof information regarding the data entity, the association of the dataentity with other entities, other data, etc., or that otherwisedescribes the data entity. State properties may vary greatly from oneembodiment to another, from one data entity to another, and from onestate to another, and the present invention is not particularly limitedwith regard thereto.

It is noted that state properties are not necessarily required to beexact representations of the properties of the data entity. Stateproperties represent the properties of the data entity at the statetime, such that the state properties are substantially similar tocorresponding properties of the data entity. For example, a stateproperty representing the color of the data entity should at leastsubstantially correspond to the actual color of the data entity, butperfect representation is not required. However, although not required,it is permissible for state properties to be exact representations ofcorresponding properties of the data entity at the state time (forexample if the color of the data entity is itself defined by a colorcode, copying the color code as digital data might constituteestablishing an exact representation of the color).

State properties for a particular state are associated with the statetime for that particular state. Thus, a state property may be consideredto be some feature of the data entity as the data entity exists at theassociated state time.

As noted state properties may vary considerably. However, typicallythough not necessarily, each state will include as one of the stateproperties thereof a spatial arrangement property for the data entity.

The particulars of spatial arrangement properties may vary, and inparticular will vary based on the nature of the data entity for whichthe state in question is established. For example, for a data entitythat represents an augmented reality entity that is applied to alocation in the physical world, the spatial arrangement property for astate might include the position of the entity with respect to thephysical world, and/or the orientation of the entity with respect to thephysical world. Similarly, for a data entity that represents a physicalobject within the physical world, the spatial arrangement property for astate also might include position and/or orientation in some globaland/or absolute coordinate system, e.g. latitude and longitude, etc.

However, spatial arrangement properties also may be determinedrelatively, compared to some point or points. For example, a spatialarrangement property might include heading and distance from someperson, from some object or terrain feature, from some reference mark,etc., with or without absolute coordinates. Likewise, a spatialarrangement property may include orientation of an entity in relativeterms, e.g. a vehicle facing toward a person or landmark, with orwithout absolute orientation information.

In addition, spatial arrangement properties may include factors such asspeed of motion (if any)—that is, a rate of change in position—directionof motion, magnitude and/or direction of acceleration, etc.

Furthermore, it should be understood that while well-defined positionand/or orientation values may be useful for certain types of dataentities, including but not limited to data entity representingwell-defined discrete objects, spatial arrangement properties may takedifferent forms. For example, for a data entity representing a region ofrain, fog, light level, etc. a spatial arrangement property for a statethereof may not be limited to a particular point, but might insteaddefine a region. Other arrangements also may be equally suitable.

It is noted that a spatial arrangement property is not required to becomprehensive or exhaustive, and need not include all available spatialarrangement information for a data entity. That is, while a particulardata entity might have both a position and an orientation, a spatialarrangement property for a state thereof may not necessarily includeboth position and orientation.

Returning to FIG. 1, and with reference to step 104B, other stateproperties may include, but are not limited to, the shape, dimensions,surface texture, and/or color of a data entity. State properties alsoare not limited only to individual features; for example, a photographicrepresentation or 3D model might be considered a state property of adata entity. Likewise, text or other non-visual data may be consideredas a state property. State properties and implications thereof arefurther described subsequently herein.

In summary, the step of establishing a state 104 for a data entityincludes establishing a state time 104A, substantially representing thetime at which the state is established 104, and establishing stateproperties 104B that describe to at least some degree the state of thedata entity as of the state time (typically but not necessarilyincluding a spatial arrangement of the data entity). A state thusrepresents in some sense a representation of a moment in time for a dataentity, with information describing the data entity at that moment, e.g.where the data entity was, how the data entity appeared, etc.

Moving on in FIG. 1, the data entity is stored 108. As the data entityis a data construct, typically though not necessarily the data entitymay be stored on some form of digital medium such as a data store, e.g.a hard drive, solid state drive, optical drive, cloud storage, etc.However, other arrangements may be equally suitable, including but notlimited to retaining the data entity in on-board memory of a processor.

The state is also stored 110. As with the data entity itself, the statetypically though not necessarily may be stored on some form of digitalmedium such as a data store, e.g. a hard drive, solid state drive,optical drive, cloud storage, etc., though other arrangements may beequally suitable, including but not limited to retaining the state inon-board memory of a processor.

Storage of the data entity and/or states therefor must be sufficient soas to enable subsequent output thereof at a subsequent output time(described in detail later herein). However, it is emphasized thatstorage is not required to permanent, nor is storage required to be ofany particular duration. Moreover, for purposes of establishing a dataentity and a state therefor, the present invention does not require thatoutput actually take place, so long as output at some output time ispracticable for the stored data entity and stored state.

Turning now to FIG. 2, an example method for outputting associatedcontent for an entity according to the present invention is showntherein.

In the method shown in FIG. 2, a data entity is received 230 in aprocessor.

In addition, a state is received 232 for the data entity. As withestablishing a state 104 in FIG. 1, receiving a state 232 in FIG. 2 mayfor illustrative purposes be considered as two sub-steps, receiving astate time 232A and receiving state properties 232B for the data entity.However, it is again emphasized that in at least some embodiments thesub-steps may be executed together, simultaneously, in parallel, etc.,and are not necessarily required to be executed as distinct steps.

A state time, as previously described with regard to FIG. 1, is a timereference for when the state is established. A state property, as alsopreviously described with regard to FIG. 1, is some feature of the dataentity itself or that is or may be associated with the data entity,and/or that provides some degree of information regarding the dataentity, the association of the data entity with other entities, otherdata, etc., or that otherwise describes the data entity.

It is not required that a state be received 232 with all of the stateproperties that were initially established for that state. While for atleast some embodiments it may be desirable to receive all availableinformation, that is, all of the state properties that were establishedfor that state, for other embodiments it may be useful to limit thenumber of state properties that are received and/or considered, forexample to reduce processing requirements, to avoid overtaxing slow dataconnections or slow storage systems, or for other reasons.

The present invention is not particularly limited with regard to how orfrom what source the data entity and/or the state is received. The dataentity might be received from another processor, from a data store suchas a hard drive or cloud storage, from a communications link such asthrough wired or wireless communication, etc. Other arrangements may beequally suitable. In certain embodiments, a data entity may beconsidered to be “received” if that data entity is generated by orwithin the processor in question itself. Just as the manner by which adata entity is established as described with respect to FIG. 1 aslimited only insofar as the need for a functional data entity to be mademanifest within a processor as described therein, likewise for thearrangement of FIG. 2 the manner by which the data entity is receivedalso is limited only insofar as the need for a functional data entity tocome to be present in some fashion within a processor.

In so far as a distinction may exist between establishing an entity anda state in as shown in FIG. 1, and receiving an entity and a state in asshown in FIG. 2, with respect to establishing a data entity and a statecontent is to be associated, i.e. the state (and thus the state time andstate properties) is being associated with the data entity, while withrespect to receiving a data entity and a state the association ofcontent is already accomplished, that is, the state is associated withthe data entity.

Returning to FIG. 2, with the data entity received 230 and the statereceived 232 an output is executed 236 at an output time. Outputting mayfor illustrative purposes be considered as two sub-steps, outputting thedata entity 236A and outputting the state properties applied thereto236B. However, with regard to outputting 236 in at least someembodiments the sub-steps may be executed together, simultaneously, inparallel, etc., and are not necessarily required to be executed asdistinct steps.

Thus, for illustrative purposes it may be considered that the dataentity is outputted 236A. That is to say, insofar as the data entityincludes therein information in a form as may be outputted, thatinformation is outputted. However, as previously described stateproperties of a data entity are part of a state, that state beingassociated with the data entity. For at least some embodiments, few ifany properties suitable for output may be part of the data entityitself. For example, if the properties desired to be outputted will bestored within the state as state properties, there may be no need toduplicate such information within the data entity proper. It is evenpossible that for at least some embodiments, the data entity may benothing more than an identifier, e.g. a name assigned to designate avirtual reality object, augmented reality object, etc., with little orno additional data incorporated therein.

Continuing in FIG. 2, at least one of the state properties is applied tothe data entity 236B and outputted. For example, for an arrangementwherein state properties include a 3D model, a position, and anorientation, the data entity may be outputted as having and/or beingrepresented by that 3D model, in that position, with that orientation.In effect, the data entity, having had applied thereto state propertiesfrom a state that was established at a state time, essentiallyrepresents the data entity as the data entity existed at the state time.The result may be considered to present a snapshot of the data entity asthe data entity existed at some moment in time.

It is not required that all of the state properties received for a statebe applied to the data entity 236B as part of output 236. While for atleast some embodiments it may be desirable to apply and/or output allavailable information, that is, all of the state properties that werereceived for the state, for other embodiments it may be useful to limitthe number of state properties that are applied and/or outputted, forexample to reduce loads on graphic processors, to avoid cluttering adisplay or other device outputting the information, or for otherreasons.

It is emphasized that the output time may be, and typically though notnecessarily will be, different from the state time. That is to say, adata entity typically will be outputted with a state associatedtherewith at a time other than the time at which that state wasestablished.

In addition, it is noted that the output time represents a time when theentity is outputted along with the state therefor. Unlike the statetime, which as part of a state is associated with a data entity (thestate time thus typically although not necessarily being stored as partof the state), the output time is not necessarily associated with thedata entity in any lasting fashion, nor necessarily with any stateassociated with the data entity. While associating an output time with astate, with a state time, and/or with a data entity is not prohibited(e.g. logging each output of the state and associating the output timeswith the state), for at least some embodiments the output time mayneither be recorded nor associated with any other data.

Moving on to FIG. 3, an example method for associating content for anentity according to the present invention and outputting associatedcontent for the entity according to the present invention is showntherein. The arrangement in FIG. 3 is at least somewhat similar to boththe arrangements shown in FIG. 1 and in FIG. 2, with FIG. 3 representinga combined approach wherein content is both associated and outputtedwith the association. It will be understood, for example in view of FIG.3, that although the establishing and storing of entities and statesaccording to the present invention and the receiving and outputting ofentities and states according to the present invention may be executedseparately, e.g. with different hardware, for different users, and/or atdifferent times, the establishing, storing, receiving, and/or outputtingof entities and states according to the present invention also may beexecuted together in a substantially continuous fashion.

In the method shown in FIG. 3, a data entity is established 302. A stateis also established 304 for the entity. As noted previously,establishing the state 304 may be considered as sub-steps ofestablishing a state time 304A and establishing state properties 304Bfor the data entity.

The data entity is stored 308. The state also is stored 310. Aspreviously noted, for at least certain embodiments the data entityand/or the state may be stored within a processor itself. Storagethereof is not limited only to hard drives, solid state drives, or otherdedicated storage devices, nor is storage required to be permanent nornecessarily even of any particular duration. Storage only requires thatthe relevant data is retained for sufficient time and in sufficientcondition as to enable output thereof.

Moving on in FIG. 3, the data entity is received 330. The state also isreceived 332. Again, receiving the state 332 may be considered as twosub-steps, receiving the state time 332A and receiving the stateproperties 332B.

Output is then executed 336 at an output time, the output time beingdifferent from the state time. As previously noted, outputting 336 maybe considered as two sub-steps, outputting the data entity 336A andapplying one or more state properties thereto 336B.

Many variations on the methods shown in FIG. 1 through FIG. 3 arepossible within the scope of the present invention. For example,multiple data entities may be associated with states, multiple statesmay be associated with a single data entity, and states may be outputtedin multiple fashions. FIG. 4 through FIG. 7 show examples of certainsuch variations, though the present invention is not limited only to thevariations shown.

With regard to FIG. 4, therein a method is shown for associating contentwith multiple entities. In the example arrangement of FIG. 4, a firstdata entity is established 402. A state is also established 404 for thefirst data entity. Establishing the state 404 may be considered assub-steps of establishing a state time 404A and establishing stateproperties 404B for the first data entity. The first data entity isstored 408, and the state for the first data entity is also stored 410.

Moving on in FIG. 4, a second data entity is established 414. While thesecond data entity may be, and typically is, a distinct data entity fromthe first data entity established in step 402, the second data entity isotherwise defined similarly. That is, the second data entity is aninformation construct the represents one or more objects, elements,phenomena, locales, environments, etc., and more particularly may be anaugmented reality or virtual reality entity. Further discussionregarding data entities is provided above with regard to FIG. 1.

Returning to FIG. 4, a state is established 416 for the second dataentity. Similarly to the state for the first data entity, establishingthe state 416 may be considered as sub-steps of establishing a statetime 416A and establishing state properties 416B for the second dataentity. Also similarly, the second data entity is stored 408, and thestate for the second data entity is also stored 410.

It should be understood that just as multiple data entities may beestablished and stored in association with states therefor, multipledata entities similarly may be received and outputted with statestherefor.

Generally, the present invention is not limited insofar as how many dataentities may be associated with states therefor. However, the examplearrangement of FIG. 4 may illuminate certain possible features of thepresent invention, related to the ability to associate multiple entitieswith states therefor.

For example, for an arrangement wherein multiple data entities areassociated with states therefor, it becomes possible to effectivelycapture states for a group of entities. Possible groups might includesimilar entities, e.g. all entities within some radius of a givenposition, all entities created by a single source, all entities with asimilar feature such as color, size, identity, etc. Other groupings arealso possible.

In particular, it is noted that multiple data entities may be associatedwith states wherein those states have substantially the same state time.That is, if a state may be considered to be a snapshot of a single dataentity at a moment in time, by associating multiple data entities withstates having substantially the same state time it becomes possible toeffectively retain snapshot of a group of entities at a moment in time.Thus, it is likewise possible to receive and output those data entitiesall with states having substantially the same state times associatedtherewith. One result of such association, storage, and output is that agroup of data entities may be portrayed as that group existed at someother moment. To continue the example above, all entities within someradius of a given position might be outputted as those entities existedat some previous moment in time.

Thus, although for simplicity the present invention is at times referredto herein as addressing an individual data entity, it should beunderstood that the present invention is not limited only to individualdata entities. Groups of entities, regions or subsets of larger virtualor augmented reality environments, or potentially even entire virtual oraugmented reality environments may be captured at a moment in time, andoutputted at some other time with a former state.

Moving on to FIG. 5, therein a method is shown for associating a dataentity with multiple states therefor. In the example arrangement of FIG.5, a data entity is established 502. A first state is established 504for the data entity. Establishing the first state 504 may be consideredas sub-steps of establishing a first state time 504A and establishingfirst state properties 504B for the data entity.

A second state is also established 506 for the data entity. Establishingthe second state 506 likewise may be considered as sub-steps ofestablishing a second state time 506A and establishing second stateproperties 506B for the data entity.

Moving on in FIG. 5, the data entity is stored 408. The first state forthe data entity is stored 410, and the second state for the data entityis also stored 512.

As may be seen from the example of FIG. 5, a given data entity may beassociated with multiple states therefor. It is emphasized that thepresent invention is not particularly limited with regard to how manystates a given date entity may have. The number of states for any dataentity may be as small as one, or may be arbitrarily large. Since eachdata state represents, in effect, the state of the data entity as of thestate time, each data state may be taken to represent the status of thedata entity at some moment in time. Multiple states thus may be taken torepresent multiple moments in time for a data entity. Alternately,multiple states for a data entity might be considered to represent ahistory for the data entity.

The ability to retain a history for data entities is one advantage ofthe present invention. It will be understood that a history need not beperfect or comprehensive in order to be useful. That is to say, notevery event, change, etc. for a data entity must be or necessarily willbe represented by a state therefor. However, representing every change,event, etc. also is not prohibited, and for at least certain embodimentsit may be advantageous to retain a complete log of changes to a dataentity according to the present invention. Such a log might beconsidered at least somewhat analogous to a wiki page history, whereineach change to the page is logged and recorded with the time the changeis made. Such functionality may be implemented through the use of thepresent invention for virtual reality and/or augmented reality entities.

Although FIG. 5 shows a single continuous arrangement wherein the secondstate is established immediately after the first state, this is done forillustrative purposes only. The present invention is not limited only toestablishing states immediately one after another, nor is there anyparticular limit regarding how much time may pass (or how many changesmay occur to the data entity) between the establishment of consecutivestates.

Furthermore, for embodiments wherein multiple data entities areassociated with states therefor (as in FIG. 4), establishing a state forone such entity does not require that a corresponding state beestablished for another such entity. Different entities are not requiredto have states with the same state times, or to have the same number ofstates. Although for certain embodiments it may be useful to regularly(e.g. at some standard interval) establish states for some or all dataentities substantially at the same state times, such an arrangement isnot required.

Turning now to FIG. 6, an example method for outputting associatedcontent for multiple states of an entity in multiple iterationsaccording to the present invention is shown therein.

In the method shown in FIG. 6, a data entity is received 630 in aprocessor.

In addition, a first state is received 632 for the data entity.Receiving the first state 632 may for illustrative purposes beconsidered as two sub-steps, receiving a first state time 632A andreceiving first state properties 632B for the data entity.

A second state also is received 634 for the data entity. Receiving thesecond state 634 also may be considered as two sub-steps, receiving asecond state time 632A and receiving second state properties 632B forthe data entity.

An output of a first iteration of the data entity is executed 636 at anoutput time. With regard to iterations, it should be understood that thedata entity, being information, may be outputted multiple times, so asto be present within a virtual or augmented reality environment asmultiple iterations of the same data entity. The first iteration of thedata entity is outputted in conjunction with the first state. Outputtingthe first iteration may be considered as two sub-steps, outputting thefirst iteration of the data entity 636A and outputting the first stateproperties applied thereto 636B.

An output of a second iteration of the data entity also is executed 638at the output time. The second iteration of the data entity is outputtedin conjunction with the second state. Outputting the second iterationalso may be considered as two sub-steps, outputting the second iterationof the data entity 636A and outputting the second state propertiesapplied thereto 636B.

It is pointed out that both the first and the second iterations areoutputted at the output time. Thus both iterations are outputtedsubstantially at the same time. The data entity is in effect outputtedas two copies thereof, in two different states: once in the first state,and once in the second state. Multiple iterations of the data entity maythus be visible within the virtual reality environment and/or augmentedreality environment. The first and second iterations may be compared,whether side-by-side, overlaid, or in some other fashion.

It will be understood that the present invention is not limited only totwo iterations of any particular data entity. An arbitrarily largenumber of data entities may potentially be outputted at any given time.

Turning now to FIG. 7, an example method for outputting associatedcontent for multiple states of an entity in a multiple iterations atdifferent times according to the present invention is shown therein.

In the method shown in FIG. 7, a data entity is received 730 in aprocessor.

A first state is received 732 for the data entity. Receiving the firststate 732 may for illustrative purposes be considered as two sub-steps,receiving a first state time 732A and receiving first state properties732B for the data entity.

A second state also is received 734 for the data entity. Receiving thesecond state 734 also may be considered as two sub-steps, receiving asecond state time 732A and receiving second state properties 732B forthe data entity.

An output of a first iteration of the data entity is executed 736 at afirst output time. The first iteration of the data entity is outputtedin conjunction with the first state. Outputting the first iteration atthe first output time may be considered as two sub-steps, outputting thefirst iteration of the data entity 736A and outputting the first stateproperties applied thereto 736B.

An output of a second iteration of the data entity also is executed 638at a second output time. The second iteration of the data entity isoutputted in conjunction with the second state. Outputting the seconditeration at the second output time also may be considered as twosub-steps, outputting the second iteration of the data entity 736A andoutputting the second state properties applied thereto 736B.

It is pointed out that the first and second iterations are outputted atfirst and second output times (as opposed to the arrangement of FIG. 6,wherein both iterations were outputted at the same output time). Thusthe two iterations are outputted substantially sequentially. The dataentity is in effect outputted as two copies thereof, in two differentstates, at two different times: once in the first state, and once in thesecond state.

To a person viewing the output, the appearance will depend on theparticulars of the states and any choices made with regard to output. Ifthe first and second iterations are outputted with the same location andorientation, and output of the first iteration stops substantially atthe time that output of the second iteration begins, the appearance to aviewer would be of the outputted data entity changing between the firstand second states. Essentially, a stationary animation of the dataentity would be outputted (albeit only a two-frame animation for onlytwo states).

However, if the first and second iterations are outputted with differentlocations and/or and orientations (for example, if the two states havedifferent spatial arrangement properties and are outputted therewith),and output of the first iteration stops substantially at the time thatoutput of the second iteration begins, a mobile animation of the dataentity would be outputted (albeit again only a two-frame animation foronly two states).

If output of the first iteration does not stop when output of the seconditeration begins, two iterations may be present at once, whetherside-by-side, overlaid, etc., in a manner similar to that described withregard to FIG. 6.

It will again be understood that the present invention is not limitedonly to two iterations of any particular data entity for the examplearrangement as described with regard to FIG. 7. An arbitrarily largenumber of data entities may potentially be outputted, for exampleproviding potentially long and/or detailed history animations of a dataentity.

Again, at this point it may be useful to elaborate on certainimplications of the functionality of the present invention. Inassociating states with data entities over a period of time, a record ofthe status of those entities over time is created. In outputting dataentities with states having state times different from the output times(e.g. different from the present), it is possible to view or otherwiseinteract with a data entity, or a group of data entities, as that dataentity or data entities was at some point in the past. The effect mightbe compared to a “virtual time machine”.

As an example, consider an arrangement of a group of virtual objects inthe vicinity of some location within a virtual or augmented realityenvironment. By establishing multiple states for the virtual objectsover a period of time, and storing those states and the virtual objects,data representing a record of those virtual objects over that period oftime is accumulated. In response to a command from a user of the virtualor augmented reality environment, and/or in response to some otherstimulus, the virtual objects may be outputted substantially in thestate that the virtual objects were in at some earlier time.

It is noted that as previously described, one of the state propertiesestablished typically is a spatial arrangement for a given data entityat a particular state time. Thus, the position, orientation, etc. of thedata entities for a given state time typically is known. As a result,data entities may be outputted not merely with properties such as theirappearance at a particular state time, but in the position, orientation,etc. that those data entities exhibited at the corresponding state time.Thus one or more data entities may be outputted substantially in thecondition those data entities exhibited, in the positions andorientation that those data entities exhibited, as of a particular statetime.

In so doing, from the perspective of the user, the user effectivelytravels back in time so far as a virtual or augmented realityenvironment is concerned. (More precisely the environment has beenreverted to a state corresponding to an earlier time, however the effectis similar so far as a viewer is concerned.) Through the use of thepresent invention, the user therefor might be considered to have the useof a virtual (and/or augment) time machine, since groups of dataentities, virtual or augmented reality scenes, even entire virtual oraugmented reality environments may be reconstructed substantially as atsome earlier moment.

However, although storing and outputting entities and associated stateswithin a region and/or with original spatial arrangements may be usefulfor certain embodiments, the present invention is not limited only tospatial groups or to original spatial arrangements. Although typicallythe spatial arrangement of a data entity is established as a stateproperty for a state thereof, the present invention does not require allstate properties to be stored or outputted. Thus, it is for examplepossible to output data entities with certain properties as those dataentities exhibited at a previous time (the state time), but notnecessarily with all such properties. As a more particular example, itis possible to output a data entity with the appearance, color, etc. asat a state time, but in a different position and/or with a differentorientation than that entity exhibited at that state time.

In addition to being outputted with different spatial arrangements,entities can be outputted at a single output time with different statetimes, with some but not all of the state properties from a particularstate, etc. The “virtual time machine” functionality thus is moreflexible than what might be considered “pure” time travel, in that thepresent invention is not limited to showing objects, scenes,environments, etc. in entirety as at some earlier time, but ratherindividual data entities may be collected, rearranged, tailored insofaras selection of properties, outputted with some properties but notothers, etc.

For example, a user might reconstruct a region of a virtual environmentas that environment existed two weeks previously (i.e. by outputting thedata entities that were present, in the state that those data entitieswere present), while the user is in that region of the virtualenvironment. However, the user might also reconstruct the same region ofthe virtual environment while the user is in a different region, orpotentially in an entirely different virtual and/or augmented realityenvironment. Moreover, the user might reconstruct individual elements ofthe from the region (i.e. some but not necessarily all data entitiestherein), with elements not originally present, with changes in theposition, orientation, and/or other properties of certain elements, etc.

It is noted that such functionality provides one example of sharpcontrast between the present invention and a “game save” function. Theterm “game save” typically implies an arrangement that is limited torestoring an entire “world” precisely as at a particular time, with novariation in what elements might be present, where those elements mightbe, what condition those elements might be in, where the user/avatar (ifany) might be, whether other users/avatars might be present, etc. Bycomparison, although the present invention enables such “no changes”restorations, the present invention is not limited only to exact statesfor entire environments, and so provides much greater flexibility. Inthe present invention, depending on the specifics of a particularembodiment, a user is instead provided with numerous options regardingwhat can be restored, how, under what conditions, and so forth,functionality entirely unlike a game save (and that functionality beingbeyond the ability of a game save to provide).

As previously noted, the range of potential features or information thatmay be established as state properties (and that may be stored,received, and outputted) is extremely large. In principle, anyinformation that describes a data entity, a behavior of a data entity, arelationship of a data entity with something else (e.g. a user, anotherdata entity, a physical object, etc.), or some other aspect of the dataentity may be established as a state property. Likewise, for dataentities that have a parent entity and/or that depict or otherwiserepresent that parent entity, any information that describes the parententity, the behavior of the parent entity, the relationship of theparent entity with something else, etc. also may be established as astate property.

State properties may, for example, include visual representations ofsome or all of a data entity, such as an image, video, 2D models, 3Dmodels, other geometry, etc. State properties may includerepresentations with respect to other senses, such as audio information,tactile information, texture, olfactory information, etc.

State properties may include features of visual or other sensoryinformation, either in addition to or in place of visual or othersensory representations. For example with regard to visual phenomenacolor, color distribution, one or more spectra or spectral signatures,brightness, brightness distribution, reflectivity, transmissivity,absorptivity, and/or surface appearance (e.g. surface texture) mightserve as state properties. Environmental conditions such as degree ofillumination, direction of illumination, color of illumination, etc.also might serve.

State properties also may encompass features of the output of a dataentity, and/or of the system outputting the data entity. For example,features such as image resolution, whether the data entity is animated,frame rate, color depth (i.e. the number of bits used to representcolor), audio sampling rate, etc. might be utilized as state properties.

Properties that are not necessarily directly perceivable by human sensesalso might be utilized as state properties. For example, temperature,temperature distribution, composition, chemical concentration,electrical potential, electrical current, mass, mass distribution,density, density distribution, and so forth may be considered as stateproperties for certain embodiments.

Moreover, information that may be considered an abstraction also mightbe considered when establishing state properties. Features such aswhether a data entity is present in a certain location, whether the dataentity is visible (e.g. resolved or outputted within the system; not alldata entities are necessarily outputted or visible at all times evenwhen technically present with the processor managing a virtual oraugmented reality environment, for example), a number of instances of adata entity that may be present (in certain virtual and/or augmentedreality environments some or all data entities may be instanced, thatis, multiple copies thereof may be outputted at once), etc. may beutilized.

One notable example of an abstraction that could be considered as astate property might be the identity of the data entity. That is to say,what is the data entity, and/or what does the data entity represent? Itwill be understood that the identity of a data entity may refer, atleast partially, to a parent entity as well; a data entity that is basedon a parent entity in the form of a physical-world chair might forcertain purposes be identified as “a data entity”, but might moreusefully be identified as “a data entity representing the chair”(perhaps with additional information to specify which physical-worldchair is being represented). Similarly, a data entity that is meant torepresent a table but that is not based on any particular parent objectmight reasonably be identified as “a data entity representing a table”(again perhaps with additional information to specify which type oftable is represented). For at least some embodiments, it may besufficient to identify such data entities as simply “representing thechair” and “representing a table”, or even “chair” and “table”, sincethe identify of a data entity as a data entity may under at least someconditions be considered to be self-evident. (For example, if a dataentity representing a chair is disposed on a digital processor, it isarguably self-evident that the data entity is indeed a data entity, orat the least that the data entity is not a physical chair that hassomehow become disposed on a digital processor.)

Another notable example of an abstraction might be to associate usefulinformation regarding a physical object with a data entity that isestablished using that physical object as a parent entity. For example,a wiring schematic for an electronic device might be established as astate property for such a data entity. In calling up the state, a usermight call up the wiring schematic, thus conveniently accessinginformation relevant to manipulation of the parent object. For certainembodiments, it may be useful to also utilize spatial arrangementproperties so as to position the wiring schematic to overlay thephysical wiring that the wiring schematic represents. Similararrangements could be made with regard to mechanical devices (e.g. forproper operation and/or for repair), for exercise equipment (e.g.presenting schematics for proper form overlaid over the equipment and/orthe user's own body or avatar), etc.

At this point it is pointed out that the ability to determine a spatialarrangement for a data entity—as previously noted with regard to stateproperties, the spatial arrangement property being typically establishedfor at least most states for at least most data entities—may for atleast some embodiments imply the ability to determine, e.g. through theuse of sensors, the position of data entities. Similarly, if such dataentities are to be established from physical-world parent entities, suchsensors and/or other means for establishing spatial arrangements mayalso be suitable for establishing spatial arrangements for physicalworld objects. Thus, with respect to the above example regardingoverlaying schematics or other information, at least certain embodimentsof the present invention may facilitate such overlay without additionalsupport. It should be understood that such ability to determine thepositions, orientations, etc. of data entities and/or physical objectscan support additional richness insofar as augmented reality, i.e.interactions between physical and non-physical objects and phenomena.

Returning to example state features, behavioral features of a dataentity may be used as state properties, such as whether a data entity ismobile, whether the data entity is in fact moving, the speed, direction,acceleration, etc. of motion (if any), whether the data entity issubject to forces such as gravity and/or to analogs of such forces,whether the data entity reacts as a solid object when touched (asopposed to allowing other objects, users, etc. to pass therethrough, forexample), etc.

Data not directly related to the outputting of a data entity also may beconsidered when establishing state properties. For example, text andnumerical data of many sorts might be utilized. More specific examplesmight include RFID data, barcode data, data files of various sorts,executable instructions, hyperlinks, data connections, and communicationlinks all might be considered.

Data for state properties is not limited only to data directlyapplicable to the data entity. For example, for arrangements wherein thedata entity in question is based on a parent entity, such as a physicalobject, data relevant to the physical object may be incorporated intostate properties regardless of whether the data may be directlyapplicable or even relevant to the data entity in and of itself. Forexample, many of the properties referred to above—for example, mass—maynot necessarily be applied to a data entity within a virtual oraugmented reality system (that is to say, a virtual object may not have“mass”, e.g. if the virtual reality system does not account for mass inhandling virtual objects therein). However, the mass of a parent objectmight nevertheless be considered in establishing a state property. Thedata entity thus might have a mass state property associated therewith,regardless of whether the data entity itself has mass or even anybehavior features analogous to mass.

In addition, still with regard to properties that do not necessarilyapply directly to the data entity itself, information other thanphysical features also may be considered when establishing stateproperties. For example, the price of a parent entity, nutritionalinformation for a parent entity, user reviews of a parent entity, etc.might be utilized as state properties.

Data relating not necessarily to a data entity itself but rather toassociations of that data entity also may be considered whenestablishing state properties. For example, who or what created and/ormodified a particular data entity, what system (i.e. name, system ID,etc.), processor, operating system, etc. a data entity was created orutilized under at the relevant state time, etc. Likewise, if the dataentity is associated with something else (e.g. another data entity) asof the state time, such an association might be considered inestablishing state properties. As an example of the previous,“ownership” of a data entity by a particular user might be consideredsuch an association. Other associations might include whether a dataentity was in contact with something else, whether the data entitycontained or was contained by something else (consider for example avirtual chest with virtual objects therein), etc. Associations couldpotentially be relatively abstract, for example “was an avatar presentwithin 50 feet at the state time?”, or even “was a specific avatarpresent and running within 50 feet at the state time?”

The present invention is not limited only to preceding example stateproperties. Other state properties and/or data therefor may be equallysuitable.

Typically though not necessarily, a state property may be defined beforedata is acquired for the property, i.e. a determination might be madethat one of the state properties for a particular data entity or groupof data entities will be a 3D model, or a color distribution, etc. Otherdefinitions might also be made, for example the degree of precision fora state property, the type of 3D model to be used, the color model usedfor a color distribution (such as RGB, CYMK, or some other model).

Regardless, any data needed to establish a state property may beacquired in many ways. The data in question could be obtained using oneor more sensors adapted to sense properties of a parent object if theparent object is a physical object, a virtual or augmented realityobject that is outputted so as to be sensible, etc. A wide variety ofsensors may be suitable, depending on the particulars of a givenembodiment, the nature of the parent object or other target to besensed, the nature of the state property, etc.

Data needed to establish a state property also might be received fromsome source, e.g. in digital form suitable for use by a processor. Suchdata might be retrieved from a data store such as a hard drive or solidstate drive, might be received from online or “cloud” storage, might beobtained via a communication link such as a wifi modem or other wirelesssystem, a data cable, etc.

In other instances data might be read from a parent entity. Forinstance, a parent entity that is a 3D virtual object might include, aspart of the data making up that virtual object, information thatdescribes the 3D shape of the virtual object. For an embodiment wheresuch data is sufficient for purposes of establishing a state property,the data might be read from the virtual object itself, obtained from aprocessor controlling the virtual object, etc.

In yet other instances data for establishing a state property might begenerated, for example inside a processor. One example wherein thismight be performed would be an arrangement wherein a virtual oraugmented entity is produced (or at least modified) by a processor; insuch case the processor may generate the data needed to create the dataentity, and that created data might also be sufficient to establish oneor more state properties that describe the data entity for the purposesof the present invention (whether or not the original intention increating the data entity was to use or support the present invention).

Further, state properties might be established by a user interactingwith a virtual reality or augmented reality environment. For example,state properties addressing information such as price, identity,reviews, etc. might be established and/or data entered therefor by auser. A user might enter a price to be associated with a data entity(such as a data entity representing a product or service available forpurchase), might identify a data entity and enter that identity as astate property (again keeping in mind that the identity of a data entitymay reference and/or include the identity of some parent entity), mightcompose and associate a review with an entity as a state propertytherefor, and so forth.

The present invention is not limited only to preceding examplearrangements described for establishing state properties. Otherarrangements for establishing state properties may be equally suitable.It should be understood that the particulars regarding how stateproperties may be established may depend to at least some degree on thespecific embodiment, the properties themselves, the data entities, etc.

As with the state properties and the approaches for establishing stateproperties, the potential applications of the present invention are manyand varied. Certain functions with regard to the “virtual time machine”functionality have already been noted, such as the ability to view dataentities, groups of data entities, areas within virtual and augmentedreality environments, etc. in previous states as at other times.However, the ability to attach certain state properties also facilitatesother functions, including but not limited to the following.

For example, users, programs running on a processor, and/or other actorsmay associate information with an existing data entity. For a dataentity representing, for example, a product or a service, a user orother actor might establish a state for the data entity and attach anannotation such as a review as a state property for that state. Otherusers might then be able to call up the state for the data entity atanother time, so as to see the review of the product or service. Suchcontent may be arbitrarily searchable, that is, a user might use asearch engine to find reviews for Italian restaurants from distant somepoint within (or even outside of) the virtual or augmented realityenvironment. Suitable indexing of data entities, states, stateproperties, etc. may facilitate such searching.

However, it is emphasized that if a review of a particular Italianrestaurant is associated with for example a data entity representing thefront door of the restaurant, a data entity representing some point onthe sidewalk in front of the restaurant, a data entity representing abillboard advertising the restaurant, etc. (i.e. as part of a statetherefor), the review—as one of the state properties—is directlyavailable by recalling a state of the relevant data entity at thatlocation. This is distinct from an abstract or arbitrary search: thepresent invention enables users to call up and interact with datarelevant to their environment by interacting with that environment (i.e.by manipulating data entities therein to access information associatedtherewith).

Thus, through associating states and state properties with data entitiesaccording to the present invention, a high level of data connection anduser interactivity is supported for virtual reality and augmentedreality environments.

With suitable data entities, and suitable states and state propertiesassociated therewith, many interconnections of information may beimplemented. In particular, it may be desirable (though not required)that ordinary users be permitted to initiate establishment of dataentities and/or states therefor. That is, the public may be permitted toadd content to the environment, and/or to associate content within theenvironment.

Users may, as noted, associate reviews or other useful information withdata entities representing (and/or otherwise associated with) products,services, locations, avatars, etc. Users might also include other sortsof information. Nearly any type of information might be so associated: adata entity representing/associated with a vehicle might have insuranceinformation associated therewith, a data entity representing/associatedwith a library book might have shelf location, borrowing history, duedates, etc.

As an alternative to establishing new states for existing data entities,for certain embodiments information might be added to a virtual oraugmented reality system according to the present invention throughestablishing new data entities. For example, with regard to thepreviously referenced restaurant reviews, rather than establishing a newstate for a point on the sidewalk in front of the restaurant a usermight establish a new data entity, that is, create the review as a dataentity unto itself. The review might then be a state property for thereview data entity. The user might also create for the data entity somedefault appearance, perhaps an icon, an image, a symbol or signaturepersonal to the user, etc.

In keeping with the notion of personalizing signatures and/or otherfeatures of data entities, it is noted that data entities may beestablished with different levels of access for different persons,groups of people, search engines, etc. A user might establish a reviewdata entity (or a review state for an existing data entity) that isaccessible only to that user, only to individuals that the userspecifies, only to persons with a certain access code, etc. Similarly, auser might establish the data review entity (or review state) so as tobe visible only to a certain person or persons; anyone not authorized tomanipulate the data entity (or state) simply would be unaware of thepresence thereof. It is noted that a review entity is an example only;other types of annotations, comments, and information might beestablished as data entities and/or as states for data entities.

Such restriction may support multiple functions, including but notlimited to privacy, security, and convenience. With regard to privacyand security, if a private or secure data entity or state cannot beseen, and/or cannot be opened or accessed, then the information thereincannot be read or copied, and cannot be modified. To continue theprevious example of the Italian restaurant, the owner thereof might forexample establish the door, posted menu, sidewalk, etc. so as to beaccessible read-only, without permitting modifications. Insofar asconvenience is concerned, it should be appreciated that as the number ofusers in a virtual or augmented reality environment increases, thenumber of user-created data entities and/or states typically willlikewise increase. While an individual user might be interested inproduct reviews by someone they know, sifting through thousands ofreviews left by many people may be less useful than having no reviews atall. Furthermore, if reviews or other annotations proliferatesufficiently, there is the potential to choke a virtual or augmentedreality to the point that efficient navigation therein and/or effectiveuse thereof become difficult or impossible.

As a further note with regard to avoiding such “clutter”, for certainembodiments it may also be useful to enable expiration times and/orother limits on the existence (or at least the usual output) of certaindata entities and/or states. For example, an annotation entity postedwithin an augmented reality environment regarding how best to avoid roadconstruction on a particular street may become pointless or confusingonce that episode of road construction is complete. The data entitymight be established with a termination time, after which the dataentity is deleted, no longer appears unless examining a state timeearlier than the termination time, etc.

Another function that may be facilitated by at least certain embodimentsof the present invention relates to an ability to sense activities. Aspreviously noted, state properties may relate to activities with regardto concerns such as “is someone running within some distance thereof?”Similarly, states and state properties may be established related towhether a particular individual, such as the user (or one user) of thevirtual or augmented reality environment, is performing some action. If,for example, a user picks up a real-world object associated with somedata entity, and the user then moves the real-world object, then a newstate might be applied to that data entity, and/or to a data entityrepresenting the user's avatar. As a more particular example, considerthat the real-world object is a product in a store. If the user picks upthe object and leaves the store, a state may be assigned to the dataentity that the object has is to be (or has been) purchased by the user.A financial transaction may be carried out on the basis of such dataassociations (assuming acquisition of suitable data for determiningwhether the user is indeed performing such actions). From the point ofview of the user, the user would be able to purchase an item merely bypicking it up and taking it, without the need to check out (since theappropriate data manipulations and associations would be handled withregard to the relevant data entities). Such an arrangement might bereferred to as “carry and pay” (or perhaps even “look and pay” forembodiments that do not require physical transport of the object by theuser; one such arrangement might include software purchase, wherein thesoftware would automatically download and no physical object would becarried or would even necessarily be physically present at all).

Other behaviors and functions also may be associated with data entities,states, and state properties, may be implemented, and the presentinvention is not particularly limited with respect thereto.

With regard to implementation, there may be many ways to implement thefunctions of the present invention as described herein, and the presentinvention is not particularly limited in that regard. For example, oneapproach might be to establish a 3D “language” for establishing,manipulating, and associating data entities, states, and stateproperties. Support for establishing states, state times, and otherstate properties associated with data entities might readily be writteninto such a language, in much the same way that functions forfacilitating convenient use of conventional websites are written intoHTML and similar web languages. However, other approaches may be equallysuitable.

Whether using such a 3D language or not, for certain embodiments it maybe advantageous to incorporate at least some portion of the statesand/or the state properties associated with a data entity into the dataentity itself. That is, the file or other construct that is the dataentity includes therein the data that makes up the states for that dataentity.

However, for other embodiments it may be advantageous if some or all ofthe states and/or the state properties are distinct and/or separate fromthe data entity. For example, the data entity might include pointers toa file or database wherein the states are stored.

Other approaches may be equally suitable.

Moving on now to FIG. 8, therein is shown a functional diagram of anexample apparatus for associating content for an entity according to thepresent invention.

The example apparatus of FIG. 8 includes a processor 850 adapted toexecute executable instructions. The invention is not particularlylimited with regard to the choice of processor 850. Suitable dataprocessors 850 include but are not limited to digital electronicmicroprocessors. Although the processor 850 is referred to herein forclarity as though it were a singular and self-contained physical device,this is not required, and other arrangements may be suitable. Forexample, the processor 850 may constitute a processing capability in anetwork, without a well-defined physical form.

The apparatus also includes several functional assemblies of executableinstructions 852, 854, and 856 instantiated on the processor 850. Thefunctional assemblies of executable instructions 852, 854, and 856include a data entity establisher 852, a state establisher 854, and astorer 856.

For convenience, these functional assemblies of executable instructions852, 854, and 856 may be referred to as programs; however, thefunctional assemblies of executable instructions 852, 854, and 856 arenot necessarily integrated programs, nor necessarily distinct from oneanother. That is to say, some or all of the functional assemblies ofexecutable instructions 852, 854, and 856 may be composed of multipleelements rather than being single integrated programs, and/or some orall may be parts of a single multi-function program rather than beingdistinct individual programs. The present invention is not particularlylimited with regard to how the functional assemblies of executableinstructions 852, 854, and 856 are arranged, assembled, configured,instantiated, etc. except as described herein. These comments shouldalso be understood to apply to other functional assemblies of executableinstructions as described below.

The apparatus as shown in FIG. 8 also includes a data store 868 incommunication with the processor 850. Suitable data stores 868 includebut are not limited to magnetic hard drives, optical drives, and solidstate memory devices. The present invention is not particularly limitedwith regard to the type of data store 868. Moreover, although the datastore 868 is referred to herein for clarity as though it were aself-contained physical device, this is not required, and the use ofnon-discrete and/or non-physical data storage such as cloud storage as adata store 868 may be equally suitable. Furthermore, the data store 868is not required to be a dedicated data storage device; data storagewithin the processor 850, data storage within some other processor orother device, etc. may be equally suitable. Also, although forconvenience the data store 868 is shown proximate the processor 850, insome embodiments the data store 868 may be a separate device, and/or maybe located at some considerable distance from the processor. Thus inpractice, the data store may include (but is not required to include)communication hardware, i.e. for communicating between the processor andsome relatively distant location wherein data may be stored (e.g. usingwifi or other wireless communication, wired communication, etc.).

With regard to functions of the functional assemblies of executableinstructions 852, 854, and 856, the data entity establisher 852 isadapted to establish one or more data entities, those data entitiesbeing augmented reality entities and/or virtual reality entities asdescribed previously herein (e.g. with respect to FIG. 1).

The state establisher 854 is adapted to establish one or more states forone or more data entities, states also having been described previouslyherein (e.g. with respect to FIG. 1). The state establisher 854 is moreparticularly adapted to establish a state time and a plurality of stateproperties that at least correspond substantially to properties of therelevant data entities. As noted previously, the state properties mayfor certain embodiments be identical to the properties of the dataentity (e.g. digital copies thereof), but this is not required.

The storer 856 is adapted to store data entities and states therefor inthe data store 868 in such manner as to enable output of the data entityexhibiting at least one state property as of a corresponding state time,at a time other than the state time. That is, the storer 856 stores suchinformation in the data store 868 so as to permit later output thereof(e.g. to a virtual reality environment and/or an augmented realityenvironment).

Turning to FIG. 9, therein is shown a functional diagram of an exampleembodiment of an apparatus for outputting associated content for anentity according to the present invention.

The example apparatus of FIG. 9 includes a processor 950 adapted toexecute executable instructions. As noted with regard to FIG. 8, theinvention is not particularly limited with regard to the choice ofprocessor 950. Suitable data processors 950 include but are not limitedto digital electronic microprocessors. Although the processor 950 isreferred to herein for clarity as though it were a singular andself-contained physical device, this is not required, and otherarrangements may be suitable. For example, the processor 950 mayconstitute a processing capability in a network, without a well-definedphysical form.

The apparatus as shown in FIG. 9 also includes a data store 968 incommunication with the processor 950. Suitable data stores 968 includebut are not limited to magnetic hard drives, optical drives, and solidstate memory devices. The present invention is not particularly limitedwith regard to the type of data store 968. Moreover, although the datastore 968 is referred to herein for clarity as though it were aself-contained physical device, this is not required, and the use ofnon-discrete and/or non-physical data storage such as cloud storage as adata store 968 may be equally suitable. Furthermore, the data store 968is not required to be a dedicated data storage device; data storagewithin the processor 950, data storage within some other processor orother device, etc. may be equally suitable. Also, although forconvenience the data store 968 is shown proximate the processor 950, insome embodiments the data store 968 may be a separate device, and/or maybe located at some considerable distance from the processor. Thus inpractice, the data store may include (but is not required to include)communication hardware, i.e. for communicating between the processor andsome relatively distant location wherein data may be stored (e.g. usingwifi, wired communication, etc.).

It is noted that the processor 950 and data store 968 of FIG. 9, adaptedfor outputting associated content for an entity according to the presentinvention, may be for at least some embodiments be substantially similaror identical to the processor 850 and data store 868 of FIG. 8, adaptedfor associating content for an entity according to the presentinvention. Indeed, as shown in subsequent figures certain processors anddata stores may support both aspects of the present invention. However,such similarities are examples only, and are not required.

Returning to FIG. 9, the apparatus further includes an output 974adapted to output a data entity, e.g. to a virtual reality environmentand/or an augmented reality environment. The present invention is notparticularly limited with regard to the output 974. A wide range ofoutputs, suitable of outputting a wide range of sensory and/or otherinformation, may be suitable. For example, video and/or graphicaloutputs may include but are not limited to light emitting diode displays(LED), organic light emitting diode displays (OLED), plasma screenpanels (PDP), liquid crystal displays (LCD), etc. Likewise, the use ofprojected or transmitted displays, where the viewed surface isessentially a passive screen for an image projected or otherwisetransmitted after being generated elsewhere, may also be suitable. Otherarrangements including but not limited to systems that display imagesdirectly onto a viewer's eyes also may be equally suitable. Eitherdigital or analog display technologies may be suitable. In particular,stereo systems adapted to output stereo data so as to present theappearance of a three dimensional environment to a user may be suitable.Other outputs, including but not limited to audio outputs, tactileand/or haptic outputs, olfactory outputs, etc. may also be suitable.Substantially any construct that can convey data may be suitable.

The apparatus of FIG. 9 also includes several functional assemblies ofexecutable instructions 960, 962, and 964 instantiated on the processor950. The functional assemblies of executable instructions 960, 962, and964 include a data entity receiver 960, a state receiver 962, and anoutputter 964.

The data entity receiver 960 is adapted to receive data entities fromthe data store 968, those data entities being augmented reality entitiesand/or virtual reality entities as described previously herein (e.g.with respect to FIG. 1). It is noted that, as previously described, thedata store 968 may vary considerably, thus the function of receiving thedata entity may encompass reading data from a hard drive, accepting datafrom a remote source, intaking data from an input device, etc.

The state receiver 962 is adapted to receive states associated with adata entity, those states including a state time and a plurality ofstate properties, with at least one of the state properties including aspatial arrangement of the data entity.

The outputter 964 is adapted for outputting the data entity to a virtualreality environment and/or an augmented reality environment via theoutput 974 at an output time that is substantially different from thestate time, with the data entity exhibiting at least one of the stateproperties associated with the state time.

With reference now to FIG. 10, a functional diagram of an exampleembodiment of an apparatus for associating content for an entity andoutputting associated content for the entity according to the presentinvention is shown therein. As may be understood from an examination ofFIG. 10 in comparison with FIG. 8 and FIG. 9, the example apparatusshown in FIG. 10 combines the functions as described individually forFIG. 8 and FIG. 9.

The example apparatus of FIG. 10 includes a processor 1050 adapted toexecute executable instructions. The apparatus also includes a datastore 1068 in communication with the processor 1050, and an output 1974in communication with the processor 1050.

The apparatus includes several functional assemblies of executableinstructions 1052, 1054, 1056, 1060, 1062, and 1064 instantiated on theprocessor 1050. The functional assemblies of executable instructions1052, 1054, 1056, 1060, 1062, and 1064 include a data entity establisher1052, a state establisher 1054, a storer 1056, a data entity receiver1060, a state receiver 1062, and an outputter 1064.

The data entity establisher 1052 is adapted to establish one or moredata entities, those data entities being augmented reality entitiesand/or virtual reality entities as described previously herein (e.g.with respect to FIG. 1).

The state establisher 1054 is adapted to establish one or more statesfor one or more data entities, states also having been describedpreviously herein (e.g. with respect to FIG. 1). The state establisher1054 is more particularly adapted to establish a state time and aplurality of state properties that at least correspond substantially toproperties of the relevant data entities.

The storer 1056 is adapted to store data entities and states therefor inthe data store 1068 in such manner as to enable output of the dataentity exhibiting at least one state property as of a correspondingstate time, at a time other than the state time.

The data entity receiver 1060 is adapted to receive data entities fromthe data store 1068, those data entities being augmented realityentities and/or virtual reality entities as described previously herein(e.g. with respect to FIG. 1).

The state receiver 1062 is adapted to receive states associated with adata entity, those states including a state time and a plurality ofstate properties, with at least one of the state properties including aspatial arrangement of the data entity.

The outputter 1064 is adapted for outputting the data entity to avirtual reality environment and/or an augmented reality environment viathe output 1074 at an output time that is substantially different fromthe state time, with the data entity exhibiting at least one of thestate properties associated with the state time.

Where the arrangement shown in FIG. 8 enables association of content,and the arrangement shown in FIG. 9, enables output of content makinguse of such association, the arrangement of FIG. 10 enables bothfunctions. While for at least some embodiments it may be useful for asingle apparatus to provide both such functions, for other embodimentsit may be equally suitable for an apparatus to support only one or theother (as shown in FIG. 8 and FIG. 9).

Turning now to FIG. 11, a functional diagram of an example embodiment ofan apparatus for associating content for an entity according to thepresent invention, with capabilities for distinguishing and identifyingcontent is shown therein.

The example apparatus of FIG. 11 includes a processor 1150 adapted toexecute executable instructions. The apparatus also includes a datastore 1168 in communication with the processor 1150. The apparatus alsoincludes several functional assemblies of executable instructions,specifically a data entity establisher 1152, a state establisher 1154,and a storer 1156 similar to those previously described herein.

In addition, the apparatus includes two further functional assemblies ofexecutable instructions 1158 and 1166, identified individually as a dataentity distinguisher 1158 and an identifier 1166.

The data entity distinguisher 1158 is adapted for distinguishing a dataentity from a larger body of information. This may be understood inconsidering, for example, an image of a physical object in front of areal-world background. Such an object might become the parent entity forestablishing a data entity. However, in order to establish a data entityfrom such an image it may be necessary to determine what part of theimage (and/or the data therein) represents the object, and whatrepresents the background. (Conversely, the background also might beestablished as a data entity, but it may still be necessary todistinguish the background from foreground objects in such a case.) Thedata entity distinguisher 1158 partitions available data, so as tofacilitate establishing one or more data entities therefrom.

In at least some embodiments a data entity distinguisher 1158 may beintegrated with a data entity establisher 1152, as the functions thereofare at least somewhat related: a data entity distinguisher 1158determines what may be established as a data entity, and the data entityestablisher 1152 then establishes a data entity therefrom.

It is emphasized that consideration of an image of a physical-worldenvironment is an example only. For at least some embodiments,distinguishing what parent objects may be used to establish dataentities, and/or distinguishing existing data entities as such withincomplex and/or changing environments, may also fall under the purview ofa data entity distinguisher 1158.

The present invention is not particularly limited with regard to what adata entity distinguisher 1158 may consider, or how a data entitydistinguisher may be implemented. For example, in an apparatus utilizingstereo image sensors to collect data regarding an environment a dataentity distinguisher 1158 might distinguish parent entities and/or dataentities from surrounding data by evaluating the distance to variouspoints, and determining whether targets are at different distances.However, this is an example only. Other approaches for executing similardistance-based distinctions—for example, using distance sensorsutilizing structured light, time of flight, etc.—may be equallysuitable. Likewise, approaches unrelated to distance may be equallysuitable.

The identifier 1166 is adapted to identify parent entities, dataentities, and/or state properties therefor. For example, for anembodiment wherein the identifier 1166 is adapted to identify parententities and/or data entities, the identifier 1166 may incorporateobject identification capabilities, so as to be able to determinewhether a parent entity is e.g. a car, a person, an apple, etc., and/orwhether a data entity represents a car, a person, an apple, etc. For anembodiment wherein the identifier 1166 is adapted to identify stateproperties, the identifier 1166 may incorporate features for identifying(for an example data entity representing a person) whether a person isrunning, standing, sitting, etc. Thus, typically although notnecessarily, an identifier 1166 will incorporate therein executableinstructions adapted for object identification, feature identification,and/or activity identification. The present invention is notparticularly limited with regard to approaches for objectidentification, feature identification, and/or activity identification,or with regard to the approaches or implementations for an identifier1166 overall.

Turning now to FIG. 12, therein is shown a functional diagram of anexample embodiment of an apparatus for associating content for an entityaccording to the present invention, with a chronometer and sensor.

The example apparatus of FIG. 12 includes a processor 1250 adapted toexecute executable instructions. The apparatus also includes a datastore 1268 in communication with the processor 1250. The apparatus alsoincludes several functional assemblies of executable instructions,specifically a data entity establisher 1252, a state establisher 1254,and a storer 1256 similar to those previously described herein.

In addition, the apparatus includes a chronometer 1270 and a sensor1272.

As previously noted, in establishing a state associated with a dataentity, a state time is to be established. The chronometer 1270 isadapted to establish a state time. Although shown as a distinct element,it is noted that for at least some embodiments the chronometer 1270 maybe integrated into the processor 1250, or into some other element of orin communication with the apparatus. Also, other arrangements forestablishing a state time than recording data from a chronometer may beequally suitable.

Also as previously noted, in establishing a state associated with a dataentity, a plurality of state properties are to be established, includinga state spatial arrangement. The sensor 1272 is adapted to establishstate properties, and/or to acquire data to support establishing stateproperties. It will be understood that the nature of a particular sensor1272 may depend to at least some degree on the specific state propertyor state properties to be established (or supported, etc.) therewith.For example, an imager might collect image data, color data, certaintypes of spatial arrangement data, etc. In addition, an imager (or othersensor) might determine a position of the apparatus at the state time,so as to facilitate determination of relative position of the dataentity and/or other information.

Sensors suitable for determining spatial arrangement may include but arenot limited to an accelerometer, a gyroscope, an imager, a stereo pairof imagers, a GPS sensor, a magnetometer, a structured light sensor, atime-of-flight sensor, an ultrasonic sensor, and/or a wireless signaltriangulation sensor (including but not limited to a wifi positioningsensor).

Other useful sensors may include but are not limited to a bar codereader, a chemical sensor, an electrical sensor, an electrical fieldsensor, a gas detector, a humidity sensor, an imager, a stereo pair ofimagers, a light sensor, a magnetic field sensor, a microphone, a motionsensor, a pressure sensor, a radar sensor, a radiation sensor, an RFIDsensor, a smoke sensor, a spectrometer, a thermal sensor, an ultrasonicsensor, and/or a vibration sensor.

Other arrangements, other sensors, and/or multiple sensors may also beequally suitable.

With regard now to FIG. 13, therein is shown a functional diagram of anexample embodiment of an apparatus for outputting associated content foran entity according to the present invention, with capabilities fordistinguishing and identifying content.

The example apparatus of FIG. 13 includes a processor 1350 adapted toexecute executable instructions. The apparatus also includes a datastore 1368 in communication with the processor 1350, and an output 1374in communication with the processor 1350. The apparatus also includesseveral functional assemblies of executable instructions, specifically adata entity receiver 1360, a state receiver 1354, and an outputter 1364similar to those previously described herein.

In addition, the apparatus includes two further functional assemblies ofexecutable instructions 1358 and 1366, identified individually as a dataentity distinguisher 1358 and an identifier 1366. The data entitydistinguisher 1358 and the identifier 1366 may be at least somewhatsimilar to corresponding elements described with regard to FIG. 11.However, as may be seen from FIG. 13, at least some portion ofdistinguishing data entities and/or some portion of identifying dataentities, parent entities, and/or state properties may be executed inconjunction with output of the data entities, in addition to and/or inplace of being executed in conjunction with establishing the dataentities. For example, entities not identified when originallyestablished might be identified when being outputted.

Now with regard to FIG. 14, a functional diagram of an exampleembodiment of an apparatus for outputting associated content for anentity according to the present invention, with a chronometer and sensoris shown therein.

The example apparatus of FIG. 14 includes a processor 1450 adapted toexecute executable instructions. The apparatus also includes a datastore 1468 in communication with the processor 1450, and an output 1474in communication with the processor 1450. The apparatus also includesseveral functional assemblies of executable instructions, specifically adata entity receiver 1460, a state receiver 1454, and an outputter 1464similar to those previously described herein.

In addition, the apparatus includes a chronometer 1470 and a sensor1472.

As previously noted, outputs are executed at output times, and moreover(for instances including but not limited to arrangements outputtingmultiple iterations of a data entity with different states) may beexecuted at multiple output times, those output times potentially havingsome particular relationship therebetween. The chronometer 1470 isadapted to provide data for managing output at output times. Althoughshown as a distinct element, for at least some embodiments thechronometer 1470 may be integrated into the processor 1450, or into someother element of or in communication with the apparatus. Also, otherarrangements for addressing output times may be equally suitable.

Also as previously noted, in outputting a data entity it may benecessary and/or useful to dispose such a data entity with some specificrelationship to at least some state properties thereof, whether thatrelationship is the same as at the state time or different therefrom.For example, outputting a data entity with either the same spatialarrangement as at a state time or with a different spatial arrangementthan that data entity had at the state time may require or at leastbenefit from an ability to sense current surroundings. Put more simply,in order position an output, it may be useful or necessary to senseposition. The sensor 1472 is adapted to coordinate output in accordancewith such needs and/or uses. For example, an imager might be used tocollect spatial arrangement data so as to facilitate suitablere-positioning and/or re-orienting of a data entity as compared with thespatial arrangement of that data entity at the state time (whether withthe same spatial arrangement or a different spatial arrangement).Likewise, an imager might be used to determine a position of theapparatus at the output time, so as to facilitate output with properspatial arrangement.

Sensors suitable for determining spatial arrangement may include but arenot limited to an accelerometer, a gyroscope, an imager, a stereo pairof imagers, a GPS sensor, a magnetometer, a structured light sensor, atime-of-flight sensor, an ultrasonic sensor, and/or a wireless signaltriangulation sensor (including but not limited to a wifi positioningsensor).

Other useful sensors may include but are not limited to a bar codereader, a chemical sensor, an electrical sensor, an electrical fieldsensor, a gas detector, a humidity sensor, an imager, a stereo pair ofimagers, a light sensor, a magnetic field sensor, a microphone, a motionsensor, a pressure sensor, a radar sensor, a radiation sensor, an RFIDsensor, a smoke sensor, a spectrometer, a thermal sensor, an ultrasonicsensor, and/or a vibration sensor.

Other arrangements, other sensors, and/or multiple sensors may also beequally suitable.

Turning to FIG. 15, therein is shown a functional diagram of an exampleembodiment of an apparatus for associating content for an entity andoutputting associated content for the entity according to the presentinvention, with capabilities for distinguishing and identifying content,and with a chronometer and sensor.

The example apparatus of FIG. 15 includes a processor 1550 adapted toexecute executable instructions. The apparatus also includes a datastore 1558 in communication with the processor 1550, and an output 1574in communication with the processor 1550. The apparatus also includesseveral functional assemblies of executable instructions, specifically adata entity establisher 1552, a state establisher 1554, a storer 1556, adata entity receiver 1560, a state receiver 1554, and an outputter 1564similar to those previously described herein.

The apparatus likewise includes two further functional assemblies ofexecutable instructions 1558 and 1566, identified individually as a dataentity distinguisher 1558 and an identifier 1566, also similar to thosepreviously described herein.

The apparatus further includes a chronometer 1570 and a sensor 1572,again similar to those previously described herein.

Functions and interrelationships of similar elements have been describedherein individually. As may be seen, all such functions may be combinedin one apparatus, for example such as the apparatus shown in FIG. 15.While not all functions described herein, or elements facilitating saidfunctions, necessarily will be or must be incorporated into allembodiments of the present invention, any or all such functions and/orelements may be incorporated into individual embodiments. Furthermore,additional elements and/or functions may be incorporated into anapparatus according to the present invention.

Turning to FIG. 16, therein is shown an example method for establishingcapabilities for associating content for an entity according to thepresent invention onto a processor. In the example method of FIG. 16, adata entity establisher is instantiated 1680 on a processor. Data entityestablishers have already been described herein with regard to thepresent invention, for example with respect to FIG. 8. The presentinvention is not particularly limited with regard to the manner by whichthe data entity establisher is instantiated 1680 (nor are otherinstantiation steps particularly limited with regard to manner). Forcertain embodiments, the data entity establisher might be instantiated1680 on a processor as a program, being loaded from a data store such asa hard drive or solid state drive, loading being executed and/ormonitored for example through an operating system. However, otherarrangements may be equally suitable.

Moving on in FIG. 16. a state establisher is instantiated 1682 onto theprocessor. State establishers have already been described herein withregard to the present invention, for example with respect to FIG. 8.

A storer also is instantiated 1684 onto the processor. Storers also havealready been described herein with regard to the present invention, forexample with respect to FIG. 8.

With the data entity establisher, state establisher, and storerinstantiated 1680, 1682, and 1684, the capabilities necessary to carryout a method for associating content for an entity according to thepresent invention, and/or to function as an apparatus for associatingcontent for an entity according to the present invention, are in place.

Turning to FIG. 17, an example method for establishing capabilities foroutputting associated content for an entity according to the presentinvention onto a processor is shown therein. A data entity receiver isinstantiated 1786 onto a processor. Data entity receivers have alreadybeen described herein with regard to the present invention, for examplewith respect to FIG. 9.

A state receiver also is instantiated 1788 onto the processor, and anoutputter further is instantiated 1790 onto the processor. Statereceivers and outputters have already been described herein with regardto the present invention, again for example with respect to FIG. 9.

With the data entity receiver, state receiver, and outputterinstantiated 1786, 1788, and 1790, the capabilities necessary to carryout a method for outputting associated content for an entity accordingto the present invention, and/or to function as an apparatus foroutputting associated content for an entity according to the presentinvention, are in place.

Now with reference to FIG. 18, an example method for establishingcapabilities for associating content for an entity and outputting theassociated content for the entity according to the present inventiononto a processor is shown therein.

In the example method of FIG. 18, a data entity establisher isinstantiated 1880 on a processor. A state establisher is instantiated1882 onto the processor. A storer also is instantiated 1684 onto theprocessor.

Moving on in FIG. 18, a data entity receiver is instantiated 1786 ontothe processor. A state receiver also is instantiated 1788 onto theprocessor, and an outputter further is instantiated 1790 onto theprocessor.

With the data entity establisher, state establisher, storer, data entityreceiver, state receiver, and outputter instantiated 1880, 1882, 1884,1886, 1888, and 1890, the capabilities necessary to carry out a methodfor associating content and outputting associated content for an entityaccording to the present invention, and/or to function as an apparatusfor associating content and outputting associated content for an entityaccording to the present invention, are in place.

Turning now to FIG. 19, an example method for establishing capabilitiesfor associating content for an entity according to the present inventiononto a processor, along with capabilities for distinguishing andidentifying content is shown therein.

Similarly to the arrangement in FIG. 16, in FIG. 19 a data entityestablisher is instantiated 1980 on a processor. A state establisher isinstantiated 1982 onto the processor. A storer also is instantiated 1984onto the processor.

In addition, a data entity distinguisher is instantiated 1996 onto theprocessor. Data entity distinguishers have already been described hereinwith regard to the present invention, for example with respect to FIG.11. An identifier also is instantiated 1998 onto the processor.Identifiers also have already been described herein with regard to thepresent invention, for example with respect to FIG. 11.

With the data entity establisher, state establisher, and storerinstantiated 1980, 1982, and 1984, the capabilities necessary to carryout a method for associating content for an entity according to thepresent invention, and/or to function as an apparatus for associatingcontent for an entity according to the present invention, are in place.Furthermore, with the data entity distinguisher and the identifierinstantiated 1996 and 1998, the capabilities necessary to distinguishdata entities and to identify data entities, parent entities, and/orstate properties according to the present invention also are in place.

Turning now to FIG. 20, therein is shown an example method forestablishing capabilities for outputting associated content for anentity according to the present invention onto a processor, along withcapabilities for distinguishing and identifying content. Similarly tothe arrangement in FIG. 17, in FIG. 20 a data entity receiver isinstantiated 2090 on a processor. A state receiver is instantiated 2092onto the processor. An outputter also is instantiated 2094 onto theprocessor.

In addition, a data entity distinguisher is instantiated 2096 onto theprocessor. An identifier also is instantiated 2098 onto the processor.

With the data entity receiver, state receiver, and outputterinstantiated 2090, 2092, and 2094, the capabilities necessary to carryout a method for outputting associated content for an entity accordingto the present invention, and/or to function as an apparatus foroutputting associated content for an entity according to the presentinvention, are in place. Furthermore, with the data entity distinguisherand the identifier instantiated 2096 and 2098, the capabilitiesnecessary to distinguish data entities and to identify data entities,parent entities, and/or state properties according to the presentinvention also are in place.

Now with respect to FIG. 21, therein is shown a perspective view of anexample embodiment of an apparatus for associating content for an entityand/or outputting associated content for the entity according to thepresent invention. An apparatus according to the present invention maytake many forms, and/or may be incorporated into many devices.Similarly, a method according to the present invention may be executedon many devices in many forms. The arrangement of FIG. 21 shows oneexample of such a form, however, the present invention is not limitedonly to such a form.

In the apparatus of FIG. 21, a processor 2150 is present therein.Although not visible in a perspective view, a data entity establisher, astate establisher, a storer, a data entity receiver, a state receiver,an outputter, a data entity distinguisher, and/or an identifier (incombinations depending on the particulars of a given embodiment) may beconsidered to be disposed on the processor 2150.

The apparatus also includes a data store 2168. Further, the apparatusincludes first and second sensors 2172A and 2172B, illustrated asimagers in a stereo configuration, though such an arrangement is anexample only and other arrangements may be equally suitable. Inaddition, the apparatus includes first and second outputs 2174A and2174B, illustrated as display screens in a stereo configuration, thoughsuch an arrangement is an example only and other arrangements may beequally suitable. No chronometer is shown, though as previously noted achronometer may for at least some embodiments be incorporated into theprocessor 2150.

In addition, the apparatus shown in FIG. 21 includes a body 2176,illustrated in the form of a frame for a head mounted display,resembling a pair of glasses. Given such a body 2176, and thearrangement shown, the sensors 2172A and 2172B are disposed so as toview substantially forward, i.e. substantially along the line of sightof a person wearing the apparatus. Depending on the particulars of thesensors 2172A and 2172B, the sensors 2172A and 2172B may provide fieldsof view substantially similar to the fields of view of a person wearingthe apparatus. Furthermore, the body 2176 is configured such that if thebody 2176 is worn, the outputs 2174A and 2174B will be disposedproximate to and substantially aligned with the wearer's eyes. Such anarrangement may be useful for at least certain embodiments of thepresent invention. However such an arrangement is an example only, andother arrangements may be equally suitable.

The above specification, examples, and data provide a completedescription of the manufacture and use of the composition of theinvention. Since many embodiments of the invention can be made withoutdeparting from the spirit and scope of the invention, the inventionresides in the claims hereinafter appended.

We claim:
 1. A method comprising: instantiating a data entityestablisher comprising executable instructions on a processor, said dataentity establisher being adapted to establish data entities;instantiating a state establisher comprising executable instructions onsaid processor, said state establisher being adapted to establish statesfor said data entities; instantiating a storer comprising executableinstructions on said processor, said storer being adapted to store saiddata entities and said states in a data store in communication with saidprocessor; instantiating an outputter comprising executable instructionson said processor, said outputter being adapted to output said dataentities and said states to an output in communication with saidprocessor; establishing a first data entity via said data entityestablisher; establishing a first state for said first data entity viasaid state establisher; establishing a second state for said first dataentity via said state establisher, said second state being distinct fromsaid first state; establishing a second data entity via said data entityestablisher, said second data entity being distinct from said first dataentity; establishing a third state for said second data entity via saidstate establisher, said third state being distinct from said first andsecond states; wherein each of said states comprises a state time and aplurality of state properties at least substantially corresponding to arespective of said data entities substantially at said state time andcomprising a state spatial arrangement thereof; wherein a first statetime of said first state is substantially different from a second statetime of said second state; wherein a third state time of said thirdstate is substantially different from said first state time of saidfirst state; storing said first and second data entities and said first,second, and third states in said data store via said storer; retrievingfrom said data store and outputting via said outputter to at least oneof an augmented reality environment and a virtual reality environment ata same output time: a first iteration of said first data entityexhibiting at least a portion of said first state and a second iterationof said first data entity exhibiting at least a portion of said secondstate, said output time being substantially different from said firstand second state times; and said first iteration of said first dataentity exhibiting said at least said portion of said first state and afirst iteration of said second data entity exhibiting at least a portionof said third state, said output time being substantially different fromsaid first and third state times.
 2. A method comprising: establishing afirst data entity via a data entity establisher instantiated on aprocessor; establishing a first state for said first data entity via astate establisher instantiated on said processor; establishing a secondstate for said first data entity via said state establisher, said secondstate being distinct from said first state; establishing a second dataentity via said data entity establisher, said second data entity beingdistinct from said first data entity; establishing a third state forsaid second data entity via said state establisher, said third statebeing distinct from said first and second states; wherein each of saidstates comprises a state time and a plurality of state properties atleast substantially corresponding to a respective of said data entitiessubstantially at said state time and comprising a state spatialarrangement thereof; wherein a first state time of said first state issubstantially different from a second state time of said second state;wherein a third state time of said third state is substantiallydifferent from said first state time of said first state; storing saidfirst and second data entities and said first, second, and third statesin a data store in communication with said processor via a storerinstantiated on said processor; retrieving from said data store andoutputting to an output in communication with said processor via anoutputter instantiated on said processor at a same output time: a firstiteration of said first data entity exhibiting at least a portion ofsaid first state and a second iteration of said first data entityexhibiting at least a portion of said second state, said output timebeing substantially different from said first and second state times;and said first iteration of said first data entity exhibiting said atleast said portion of said first state and a first iteration of saidsecond data entity exhibiting at least a portion of said third state,said output time being substantially different from said first and thirdstate times.
 3. The method of claim 2, wherein: establishing said firstand second data entities comprises selecting said first and second dataentities, said first and second data entities being in existence priorto a selection thereof.
 4. The method of claim 2, wherein: establishingsaid first and second data entities comprises creating said first andsecond data entities.
 5. The method of claim 2, wherein: establishingsaid first data entity comprises generating said first data entity froma first parent entity; and establishing said second data entitycomprises generating said second data entity from a second parententity.
 6. The method of claim 5, wherein: said first parent entitycomprises at least one of a group consisting of a physical object, aphysical background, a physical environment, a physical creature, and aphysical phenomenon; and said second parent entity comprises at leastone of said group consisting of said physical object, said physicalbackground, said physical environment, said physical creature, and saidphysical phenomenon.
 7. The method of claim 6, wherein: generating saidfirst data entity from said first parent entity comprises sensing saidfirst parent entity with a sensor in communication with said processor;and generating said second data entity from said second parent entitycomprises sensing said second parent entity with said sensor.
 8. Themethod of claim 5, wherein: said first parent entity comprises at leastone of a group consisting of an augmented reality object and a virtualreality object; and said second parent entity comprises at least one ofsaid group consisting of said augmented reality object and said virtualreality object.
 9. The method of claim 2, wherein: at least a portion ofeach said state is incorporated within a respective of said dataentities.
 10. The method of claim 2, wherein: at least a portion of eachsaid state is distinct from said data entities.
 11. The method of claim2, wherein: at least one of said state properties comprises an identityof a respective of said data entities.
 12. The method of claim 2,wherein: said state spatial arrangement comprises at least one of agroup consisting of an absolute position of a respective of said dataentities and an absolute orientation of said respective data entity. 13.The method of claim 2, wherein: said state spatial arrangement comprisesat least one of a group consisting of a relative position of arespective of said data entity and a relative orientation of saidrespective data entity.
 14. The method of claim 2, wherein: saidplurality of state properties comprises at least one of a groupconsisting of a still image, a video, audio, olfactory data, a 2D model,a 3D model, text, numerical data, an environmental condition, animation,resolution, frame rate, bit depth, sampling rate, color, colordistribution, spectral signature, brightness, brightness distribution,reflectivity, transmissivity, absorptivity, surface texture, geometry,mobility, motion, speed, direction, acceleration, temperature,temperature distribution, composition, chemical concentration,electrical potential, electrical current, mass, mass distribution,density, density distribution, price, quantity, nutritional information,user review, presence, visibility, RFID data, barcode data, a file,executable instructions, a hyperlink, a data connection, a communicationlink, contents, association, creator, and system ID.
 15. The method ofclaim 2, wherein: establishing said first and second data entitiescomprises distinguishing said first and data entities from surroundingsthereof.
 16. The method of claim 2, wherein: said first data entitycomprises a graphical entity; said second data entity comprises a secondgraphical entity; and said output is a graphical display.
 17. The methodof claim 2, wherein: said processor, said data store, and said outputare disposed on a head mounted display.
 18. The method of claim 7,wherein: said processor, said data store, said output, and said sensorare disposed on a head mounted display.